Repository navigation
fix(sdk/go): use the CLI oidc_token.json format - #4830
TaylorMutch merged 1 commit into
Conversation
|
Hi @TaylorMutch, would you please take a look at this PR when you have a moment? I'd like to hear your thoughts as you triaged it. Thanks in advance! |
b752b4f to
86d7f48
Compare
rhuss
left a comment
There was a problem hiding this comment.
cc-review Summary
What Went Well
- Genuine cross-language anti-drift: the single fixture
sdk/conformance/oidc-token-bundle.jsonis asserted from both Rust (oidc_token.rs:197) and Go (readoidc/token_test.go:28, write:43), so a field change on either side breaks that language's test. internal/oidctokencentralizes the on-diskBundletype and converters, removing the two divergent private structs (the root-cause fix for #4508).- Missing/null
expires_atis handled consistently as non-expiring on every path (Go and Rust), with dedicated nil-vs-zero tests. - The
RefreshableTokenbackoff path is nil-error safe:nextRetry/lastErrare set together on failure and cleared together on success. - Go token persistence is atomic and 0600 (
CreateTemp+Chmod+Sync+Rename) with temp-file cleanup on every error path.
Findings
| Severity | File | Description | Source |
|---|---|---|---|
| Notable | sdk/go/openshell/v1/auth_refresh.go:140 | Fail-closed RefreshableToken change applies to every token source | goal-alignment |
| Notable | sdk/go/openshell/v1/gateway/token.go:120 | Upgrade invalidates older Go-SDK-written token files (missing issuer/client_id) | production |
| Notable | sdk/go/openshell/v1/gateway/token.go:120 | Runtime path does not match stored issuer/client_id against gateway config | security |
| Minor | sdk/go/openshell/v1/gateway/token.go:120 | Load-bearing issuer/client_id guard is undocumented | gator |
| Minor | crates/openshell-bootstrap/src/oidc_token.rs:54 | Rust writes oidc_token.json non-atomically (perm window + partial-read race) | security |
| Notable | sdk/go/openshell/v1/auth_refresh.go:82 | No-expiry cached token is never re-read (won't pick up a later expiring token) | gator |
Two findings are summary-only because they fall on lines outside this PR's diff:
- crates/openshell-bootstrap/src/oidc_token.rs:54 (Minor, pre-existing
store_oidc_token): non-atomicstd::fs::write+ separate chmod 0600 leaves a brief wider-permission window and a truncating in-place write that the Go reader can observe mid-write. The Go writer this PR adds does it correctly. In the issue's stated scope; mitigated by the 0700 parent dir and the cached-token fallback. - sdk/go/openshell/v1/auth_refresh.go:82 (Notable): a no-expiry cached token makes
isTokenValid()always true, so a long-lived process never re-reads the file and won't pick up a later CLI-written expiring token until restart. Consistent with the Rust side; the only defensible behavior without expiry metadata.
Review Details
- Findings posted: 6 (4 inline, 2 summary-only)
- Gate outcome: PASS (0 Critical, 0 Important)
- Participating reviewers: goal-alignment, production, security, gator (architecture, correctness, test-quality ran with no surviving findings; codex and coderabbit skipped on this fork PR)
|
Re: review comments in the body
I'll keep this for #4514 with the locking. I checked it's a small
Re-reading those breaks |
86d7f48 to
6054443
Compare
|
Appreciate your review @rhuss! I've replied to each review comment from my perspective. Please let me know what you think. I'm open to addressing your feedback, as you write the Go sdk mostly :) |
There was a problem hiding this comment.
Here's is another round of reviews on the latest changes (with updated agents, so there might be findings that the first round hasn't found). Although there are some findings (hopefully easy to fix by aganets), I think we are close to converge.
cc-review Summary
New round with fresh agent eyes on 6054443. The earlier threads are either addressed or continue in their existing threads (including the open issuer/client_id matching question), so this round lists only new observations. The findings come from an agent-assisted review; the failure scenarios were derived from the code and not executed.
What Went Well
sdk/go/openshell/v1/internal/oidctoken/oidctoken.go: one internalBundletype withFromToken/Tokenreplaces the two divergentoidcBundlecopies ingateway/token.goandoidc/token.go. Its field tags (omitemptyonrefresh_token,*uint64expires_at, requiredissuer/client_id) mirror the serde attributes ofOidcTokenBundleinopenshell-bootstrap.sdk/conformance/oidc-token-bundle.jsonis checked from both sides: Rustconformance_fixture_matches_cli_serialization(crates/openshell-bootstrap/src/oidc_token.rs) and GoTestWriteToken_MatchesConformanceFixtureandTestReadToken_ConformanceFixture, so a format drift between CLI and SDK fails CI.sdk/go/openshell/v1/gateway/token.godiskTokenSource: bundles withoutissuer/client_idare rejected instead of being treated as never-expiring, the expiry check fails closed at the exact expiry instant, andloadErrorgives every failure branch theErrTokenLoadwrap and theopenshell gateway login <name>hint without putting token material in the message.sdk/go/openshell/v1/auth_refresh.gousableCached/lastErr: after a failed refresh, an expired cached bearer is no longer sent, and the backoff path returns the real refresh error.lastErris read and written underr.muand cleared on success.TestGetRequestMetadata_RefreshFailureWithExpiredCachedTokenReturnsErrorcovers it withrequire.ErrorIs, a check that no "using cached token" log was emitted, and a source call count.sdk/go/openshell/v1/oidc/oidc.goLoginreuses a stored token only when the normalized issuer and the client ID match, andTestLogin_ReplacesMismatchedStoredTokencovers the client-id, issuer, and legacy-format cases in one table.TestDiskTokenSource_ExpiredTokenandTestDiskTokenSource_NoExpirytable-test the boundary valuesexpires_at=0, an absent field, and explicit null.
Findings
| Severity | File | Description | Source |
|---|---|---|---|
| Minor | sdk/go/openshell/v1/auth_refresh.go:102 | During backoff the cached lastErr is replayed, so a fresh login is not picked up for up to 30s |
gator |
| Minor | sdk/go/openshell/v1/gateway/gateway_test.go:93 | TestNewClient_AuthModeOIDC seeds a token file in a format the SDK now rejects |
test-quality |
| Minor | sdk/go/openshell/v1/gateway/gateway_test.go:345 | No gateway-level test for the ErrTokenLoad plus login hint path through resolveAuthProvider |
test-quality |
| Minor | sdk/go/openshell/v1/oidc/oidc_test.go:204 | TestLogin_ReplacesMismatchedStoredToken asserts only access_token on disk |
test-quality |
| Minor | sdk/go/docs/src/api/gateway.md:160 | ErrTokenLoad row and related comments do not mention expired tokens |
architecture |
| Notable | sdk/go/openshell/v1/gateway/token.go:107 | Reading and parsing oidc_token.json is still duplicated in the gateway and oidc packages |
architecture |
Not Posted Inline
sdk/go/docs/src/api/gateway.md:160: Location is outside the PR diff (line 160).
Minor: The ErrTokenLoad row in the error table still says "Token file missing or unreadable", and so does gateway/doc.go:64. The new paragraph right above the table (lines 151-154) lists a missing, malformed, or expired oidc_token.json, and the comment in gateway/errors.go:19-20 says "missing, unreadable, or malformed". None of these reference spots mention expired tokens.
Why this matters: The table introduces errors.Is classification, so a reader could wrap an RPC in errors.Is(err, gateway.ErrTokenLoad) to catch an expired OIDC token. That check never matches: the sentinel comes from GetRequestMetadata, gRPC turns it into a status, and the SDK StatusError does not keep the wrap chain. The paragraph above already points to v1.IsUnauthenticated, so this is about keeping the reference row in line with it. Expired tokens returned ErrTokenLoad before this PR too, so the wording gap is older than this change.
Suggested fix: In the table and in doc.go, either drop ErrTokenLoad or describe it as "returned by the auth provider's GetRequestMetadata for missing, malformed, expired, or pre-CLI-format token files; on RPCs it surfaces as Unauthenticated". Add the expired case to the errors.go comment.
Source: architecture agent
Review Details
- Findings posted: 6
- Findings reviewed and not posted: 1
- Gate outcome: PASS
- Participating reviewers: architecture, gator, production, test-quality
Signed-off-by: Yuedong Wu <dwcn22@outlook.com>
6054443 to
edfcb47
Compare
|
Heads-up: the Go SDK moves to ai-openshell/sdk-go on 2026-11-03 (details in #5031). This PR also changes |
|
Hi @rhuss, when you get a chance, could you take a relook at this PR? I think we're close to merge. Thanks for the heads-up! |
|
/ok to test edfcb47 |
Compatibility review (advisory)New or updated breaking changes; see the full report for all findings. 1. state — No explicit risk acceptance found
2. go-sdk — No explicit risk acceptance found
Review details
Advisory review; publication gates are unchanged. |
|
I'm not fully sure about the pre-condition for when a E2E tests run:
What does this |
|
Follow-up on my review: I checked the current head (
Nothing blocking from my side. Ready for a maintainer review. |
TaylorMutch
left a comment
There was a problem hiding this comment.
Reviewed CLI token format compatibility, cached-login validation, expiry handling, and refresh failure backoff. No actionable findings. The affected Go packages passed with the race detector, and required CI checks are passing.
…k-go (#4) * chore(hack): add migration and automation script scaffolding Add the hack/migration and hack/automation directories with their READMEs, the filter-repo message rules, and the history extraction and tree verification scripts used for the import from NVIDIA/OpenShell. Signed-off-by: Roland Huß <rhuss@redhat.com> * feat(hack): add port-pr.sh to move open core PRs with their authorship Formats the sdk/go commits of an open NVIDIA/OpenShell PR as patches, rewrites the module path inside them, and applies them with git am on port/<n>. Authors, dates, and the original Signed-off-by trailers are kept; the porter's format.signoff setting is ignored. Lines a 3-way merge leaves with the old path are rewritten, lines already in the base are not. Pushing and opening the PR needs --push. Signed-off-by: Roland Huß <rhuss@redhat.com> * chore(hack): record the freeze commit and v0.1.7 as the first core pin Signed-off-by: Roland Huß <rhuss@redhat.com> * refactor!: move the module path to github.com/ai-openshell/sdk-go Rewrite every github.com/NVIDIA/OpenShell/sdk/go import, the go.mod module line, the buf M options, and the docs links. Signed-off-by: Roland Huß <rhuss@redhat.com> * build: generate bindings from a pinned core tag and make mise self-contained buf.gen.yaml reads proto/ from NVIDIA/OpenShell at the tag in inputs[0].tag (v0.1.6), so no core checkout is needed. mise.toml carries its own tools and lockfile and adds format:check and core:version. Regenerating at v0.1.6 changes no bindings. Signed-off-by: Roland Huß <rhuss@redhat.com> * docs: point the README and license at the new repository Add the go get line, a supported core versions note, and a migration section for consumers of github.com/NVIDIA/OpenShell/sdk/go. Docs links move to https://ai-openshell.github.io/sdk-go/. Copy LICENSE from core and ignore coverage.out and the built book. Signed-off-by: Roland Huß <rhuss@redhat.com> * ci: run the mise CI pipeline on pull requests and main Signed-off-by: Roland Huß <rhuss@redhat.com> * test: vendor the shared conformance fixtures The OAuth client-credentials and OIDC token-bundle fixtures live in core's sdk/conformance/, which the tests reached through ../../../.. paths. Copy them next to the tests that read them, and check in conformance:check that they match core at the pinned tag. A fixture the pinned tag does not have yet (oidc-token-bundle.json came with NVIDIA/OpenShell#4830 after v0.1.7) is reported and skipped until the pin includes it. Signed-off-by: Roland Huß <rhuss@redhat.com> * build: pin core v0.1.7 v0.1.7 is the stable core release at the freeze. Its proto/ equals proto/ at the freeze commit, so the regenerated bindings do not change. Signed-off-by: Roland Huß <rhuss@redhat.com> --------- Signed-off-by: Roland Huß <rhuss@redhat.com>
Summary
The Go SDK now reads and writes
oidc_token.jsonin the CLI's format (expires_at,issuer,client_id), so one login works for both the CLI and Go programs.gateway.NewClientstops sending an expired token and fails with arun `openshell gateway login <name>`hint instead.Related Issue
Closes #4508
Changes
oidctoken.Bundlemirrors the CLI'sOidcTokenBundleand replaces the two private Go structs.gatewayrejects token files that are expired or lackissuerorclient_id, including files from earlier Go SDK builds. Every load error carries the login hint.oidc.Loginwrites the CLI format and reuses a stored token only for the same issuer and client ID.v1.RefreshableTokenno longer returns an expired cached token after a failed refresh. It returns the refresh error, which surfaces asUnauthenticated. This applies to every token source and matches the CLI.sdk/conformance/oidc-token-bundle.json. A Rust test checks it against the CLI's serialization, and Go tests check reading and writing it.Not covered: Go
Loginstill ignores the gateway'soidc_scopesandoidc_audience(#4513). Refresh and file locking stay in #4514.Release Notes
The Go SDK now shares
oidc_token.jsonwith the CLI, andv1.RefreshableTokenstops returning an expired cached token for any token source; runopenshell gateway loginonce to replace token files from earlier releases.Testing
mise run go:ci,cargo test -p openshell-bootstrap --lib oidc_token,cargo fmtandcargo clippyforopenshell-bootstrap, andmise run license:checkKey tests: the shared fixture in Rust and Go, the issue reproduction (an expired CLI-written file fails with the hint), and
RefreshableTokenfailing closed during backoff.Checklist